Geometric Analysis of Hilbert-Schmidt Independence Criterion Based ICA Contrast Function

Since the success of Independent Component Analysis (ICA) for solving the Blind Source Separation (BSS) problem [1, 2], ICA has received considerable attention in numerous areas, such as signal processing, statistical modeling, and unsupervised learning. The performance of ICA algorithms depends significantly on the choice of the contrast function measuring statistical independence of signals and on the appropriate optimisation technique. From an independence criterion’s point of view, there exist numerous parametric and nonparametric approaches for designing ICA contrast functions. It has been well known that parametric ICA methods are rather limited to particular families of sources [3]. For these parametric approaches, contrast functions are selected according to certain hypothetical distributions (probability density functions) of the sources by a single fixed nonlinear function. In practical applications, however, the distributions of the sources are unknown, and even can not be approximated by a single function. Therefore parametric ICA methods have their fatal weakness in handling many real applications. It is well known that nonparametric methods have their capability and robustness of estimating unknown distributions of the sources. Recently there have been many interests in designing nonparametric ICA contrast function. One of the possibilities is to use kernel density estimation to deal with the unknown source distributions, such as [4, 5]. There also exist other nonparametric ICA methods, which do not work with the probability density estimator directly, such as [6–8]. Most recently, the so-called Hilbert-Schmidt Independence Criterion (HSIC) was proposed for measuring statistical independence between two random variables [9]. In the sequel, an HSIC based ICA contrast has